1. Field of the Invention
This invention relates to a method and apparatus for overlaying one video signal onto another video signal. More particularly, it relates to a system for combining a motion video signal from one source and an analog RGB signal from another source, such as a graphics signal generated from a computer as might be required in a personal computer-based video conferencing environment.
2. General Background
Recently, demand has increased for improved and convenient video conferencing systems. In response to such demand, personal computer-based video conferencing systems which offer real-time, point-to-point video conferencing, have recently been proposed, including such video conferencing systems which not only display the motion video images of the video conferencing participants, but which allow for the collaboration and sharing of data and computer programs between such participants. Such systems generally communicate via an integrated services digital network (ISDN), local area network, or even over the plain old telephone system (POTS).
Basically, in order to effect point-to-point video conferencing, an incoming motion video image from the non-local (far-side) party, must be decoded, converted to a video signal, and combined with a video signal from a local (near-side) camera. In addition, in the case of computer-based video conferencing, such video images must be further processed so that they may be combined with computer graphics, such as a user interface which controls the video conference session. Furthermore, in the computer-based video conferencing scenario, the computer is not only responsible for controlling the video conference session, but is also responsible for providing data, programs, and other material that is shared with the far-side party. Many of the aforementioned computer systems accomplish this result by the addition of internal circuitry or logic cards which accomplish the combination of the motion video and graphics directly.
FIGS. 1, 2 and 3 illustrate a display of the various types of images generated during a typical computer-based video conferencing session. More specifically, FIG. 1 illustrates a combined, or composite, image which might be displayed on the monitor of the person or person operating the video conferencing session on the local, or near-side, station who is communicating with another set of people who are located at a remote, or far-side, station. As seen in FIG. 1, preferably four windows are displayed, each containing different images which are either generated by the computer or by the various components of a video conferencing system. The window labeled 2 is the user""s control center for initiating or terminating a video call or other functions as required for video conferencing, and is generated by a video conference control application or similar user interface program running on the computer on the near-side. As with other windows based applications, the video conference operator merely clicks on the desired task in order to effect that task. The window labeled 3 contains information generated by an application such as a spreadsheet, database or similar application, which is also running on the computer at the near-side. The data or other information contained therein may or may not be shared with the video conference participants on the far-side. Windows 4 and 5 are actually each composed of two components. More particularly, the window xe2x80x9cframesxe2x80x9d 6 are generated by the aforementioned video conference control application, while the content of these windows are created in the components of the video conferencing system. Near-side window 4 displays the image received at the local camera; e.g., the local video conference participant(s). The far-side window 5 displays the image received at the remote camera; e.g., the remote video conference participant(s).
FIG. 2 shows more clearly the image(s) created by the local computer. As seen therein, the contents of the near- and far-side windows are blank. (Although they are depicted in FIG. 2 as black, it will be appreciated that the contents may be arbitrary). On the other hand, FIG. 3 shows the image(s) created by the components of the video conferencing system; i.e., the images received by the near- and far-side cameras. When the above images are combined or xe2x80x9cmultiplexedxe2x80x9d, the resulting display is as shown in FIG. 3.
One computer-based video conferencing system is described in detail in U.S. Pat. No. 5,506,954, entitled xe2x80x9cPC-Based Conferencing Systemxe2x80x9d (Arshi et al). In the system shown in Arshi, real-time audio, video and data conferencing is effected in non-real-time xe2x80x9cwindowsxe2x80x9d environment. In order to accomplish such conferencing, analog video signals from a camera are first digitized by a video board, then decoded by a video capture module and compressed via video microcode running on a pixel processor. After storing the resulting compressed video in VRAM, the system""s ISA bus interface transmits the compressed video to a host interface running on the host processor, which in turn, passes the compressed video on to a video manager. After time-stamping the compressed video for synchronization with the audio signals, the compressed video is transmitted to a communications manager, which in turn passes the compressed video through a digital signal processing (DSP) interface to an Industrial Standard Architecture (ISA) bus interface of an audio/communications board which stores the compressed video in a memory. Once the compressed video is thus stored, it is formatted for ISDN transmission to a remote site.
Prior art systems such as discussed above have several disadvantages. For example, in order to effect computer-based video conferencing, such systems generally require modification to and/or addition of existing computer hardware and/or software, which in turn requires that a user open and remove the computer case. This is inconvenient and costly to the user, and subjects the computer to a number of extraneous problems. Although an alternative to opening the computer case would be to use an xe2x80x9cadd inxe2x80x9d product, such as a standard PCI bus plug-in card, most current xe2x80x9cadd inxe2x80x9d products require the use of a significant portion of the computer""s resources and computation capability. This can impair the ability of the computer to share applications with another party to the video conference session; i.e., collaborative data conferencing. Therefore xe2x80x9cadd inxe2x80x9d products are not a viable alternative. Another problem which occurs with modification of existing computer hardware and/or software is that such modifications might negatively impact the possibility for future modifications and enhancements to the computer. For example, changes to the system""s IRQ or I/O are often required when implementing a video conferencing system, and such changes may impede or even preclude future changes and/or upgrades to the system which may be necessary to keep the computer viable in a given technological environment. Finally, it is desirable to provide a video conferencing system which can be used almost universally with most computers. For example, laptops which require high quality conferencing, network computers (NC), older, less powerful computers, and computers and terminals which come in a closed (i.e., sealed) case, are precluded from being used for video conferencing purposes when the video conferencing must be added directly to the computer. Effecting the video conferencing process outside of the case of the computing device allows the range of computing devices to be considerably broadened so as to include these aforementioned devices among others.
It would be desirable therefore, to provide a system for overlaying motion video signals on computer graphics such as would be used in a real-time computer video conferencing system, wherein such system would be operable within most computers, including laptops, network computers, and sealed computers, but would not require modification of existing computer hardware and software, and thus would not necessitate increased cost and added inconvenience to the user.
Accordingly, it is one object of the present invention to provide a system and method for overlaying a motion video signal onto an analog signal.
It is another object of the present invention to provide a system and method for combining a video signal from a video source with an analog RGB signal, such as that generated by a computer.
It is yet another object of the present invention to provide a system and method for overlaying a motion video signal onto computer-generated graphics.
It is still another object of the invention to provide a computer-based video conferencing system which works universally with any computer, including laptops, is easy to install, and which does not require modification of the computer""s preexisting computer hardware and/or software.
Another object of the invention is to provide a personal computer-based video conferencing system wherein the graphics generated from the computer are directly combined with motion video signals from the separate video conferencing apparatus.
Still another object of the invention is to provide a personal computer-based video conferencing system wherein the graphics generated from the computer are directly combined with motion video signals from the near-side and far-side cameras.
Therefore, in accordance with one aspect of the invention, a system for overlaying a motion video signal onto an analog video signal on a display is provided, wherein the system includes a motion video processing unit for receiving and processing the motion video signal from a motion video source into a signal having an analog format and a video format analyzer and synchronizer device for receiving the analog video signal and for determining video timing parameters of the analog signal and for controlling video timing parameters of the motion video signal to match the video timing parameters of the analog video signal determined by the video format analyzer and synchronizer device so as to provide an output motion video signal which is synchronized with the analog video signal. The system of the present invention further determines the display of the analog video output signal or the synchronized output motion video signal on the display.
According to another aspect of the present invention, the video format analyzer and synchronizer device of the present invention preferably includes a clock signal device, a programmable frame controller, and a video format analyzer. The clock signal device provides a synthesized pixel clock which is used to regenerate the original pixel clock of the analog video signal, and thus for synchronizing the motion video signal and the analog video signal. As the frequency of the synthesized pixel clock frequently exceeds the frequency capability of many common logic devices, the synthesized pixel clock is divided to a lower frequency signal, referred to as the xe2x80x9cGroup Clockxe2x80x9d (GCLK) signal. This GCLK signal is used by the programmable frame controller as its clock signal, and it is the frame controller clock signal which is used for referencing all timing parameters (video envelope) of the resulting signal which is output to the display. Using programmable integers Htotal and Vtotal, the programmable frame controller creates xe2x80x9cinternalxe2x80x9d Hsync and Vsync signals to be used in synchronization of the motion video signal with the xe2x80x9cexternalxe2x80x9d Hsync and Vsync components of the analog video signal.
According to yet another aspect of the invention, a primary purpose of the video format analyzer is to analyze the analog video signal, in order to determine the timing parameters (video envelope) of such signal. Preferably, the video format analyzer includes a counter, for measuring the various horizontal and vertical parameters of the analog video signal. The video format analyzer begins by measuring the various vertical timing parameters, including, the start and end of active video (vertical), and the interval between certain events to determine vertical front porch, vertical back porch, and vertical active video. All of these vertical measurements are made in units of horizontal sync pulses (or horizontal scan lines). In this way the vertical resolution of the source of the computer video signal 12 is accurately determined. Similarly, the video format analyzer then measures the various horizontal timing parameters of the analog video signal, including the start and end of active video (horizontal), and the interval between events to determine horizontal front porch, horizontal back porch, and horizontal active video, in order to determine horizontal resolution. Unlike the vertical timing parameters, all of the horizontal measurements are made using the GCLK signal as the unit of measurement. The GCLK signal is a function of both the Hsync component of the analog video signal and the programmable integer Htotal. While Hsync is measured by the video format analyzer, accurate valuation of Htotal is desired and thus such value is determined by reference to a preselected horizontal resolution value preferably stored in a look-up table in memory and derived therefrom. If the derived value for Htotal is not consistent with the expected horizontal resolution, Htotal is adjusted (and thus the frequency of the synthesized pixel clock) until the measured horizontal resolution of the analog video signal is equal to its expected horizontal resolution. This ensures a correctly synthesized pixel clock.
According to still another aspect of a preferred embodiment of the present invention, in order for the video format analyzer to locate and measure the timing parameters of the analog video signal, the video format analyzer must be presented with video having known video content. The system of the present invention is preferably communicatively coupled to a computer which includes a mechanism for generating video having known video content. The known video generating mechanism preferably includes software in the computer for causing a video driver to generate a preselected number of preferably blue pixels (although red or green, or combination thereof, could be used), which are used by the system to detect the beginning and end of the active video time of the analog signal, and thus for determining the position and duration of the active video time of the analog signal so that this may be used to control the active video time of the motion video signal.
In another aspect of the present invention, the unit for determining the display of the i;o analog signal or the synchronized motion video signal on the display combines the signals and determines for a preselected number of pixels of each of the analog signal or the synchronized motion video signal, which of the two signals will be displayed. The display determining unit includes a video switch and a source controller. The video switch combines the signals and sources, or switches, the preselected number of pixels of the analog signal or the synchronized motion video signal to the display. This is effected under the control of the source controller, which is preferably a multiplexer control mask, having a memory for storing a plurality of values, wherein each value output from the memory of the multiplexer control mask controls the sourcing of the preselected number of pixels by the video multiplexer to the display. In a preferred embodiment, the system of the present invention is communicatively coupled to a computer having a user controllable video conference control application or other such user interface program, such application causing the display of xe2x80x9cwindowsxe2x80x9d in which the motion video signal will be displayed. The application would allow the user of the system to resize or move a window in a manner similar to the control of other xe2x80x9cwindowsxe2x80x9d, and would send back to the multiplexer control mask for storing in its memory, information reflecting these changes.
In addition, according to another aspect of the present invention, the video format analyzer can be used to determine whether the video is interlaced or non-interlaced. In non-interlaced video, the time difference between the leading edge of Vsync and the leading edge of Hsync of the analog video signal will be constant for consecutive measurements of these events. If the video is interlaced, this difference varies by half a scan line with each Vsync. If interlaced video is detected, the condition is flagged and operation may be aborted, if desired.
In accordance with yet another aspect of the present invention, the system further includes logic for generating an interrupt if the vertical sync pulse of the motion video signal is not synchronized with the vertical sync pulse of the analog signal. The video format analyzer may be programmed to prevent the display determining unit from displaying the motion video signal on the display until the vertical sync pulse of the motion video signal becomes synchronized with the vertical sync pulse of the analog signal. Alternatively, the video format analyzer can be programmed to allow the display determining unit to continue to display the motion video signal on the display even if vertical synchronization between the motion video signal and analog signal is not maintained.
The features of the present invention believed to be novel are set forth with particularity in the appended claims. However, the invention itself may be best understood with reference to the following description in conjunction with the accompanying drawings.